Method for restoration of blurred barcode images

ABSTRACT

The present invention relates to the field of human interaction with mobile devices, more specifically the interaction done by the automatic recognition of barcode images captured by the camera integrated in portable devices, particularly mobile phones, smartphones, Personal Digital Assistants PDAs, portable digital camera, among others. 
     The present patent application relates to a method to effect the reading of barcodes by means of portable devices with digital cameras. The present invention provides a method to restore barcode images damaged by the blurring effect due to the lack of focus of the captured images with low cost cameras with fixed focus. The proposed method treats the image of barcodes in order to enable correct decoding of its content.

FIELD OF THE INVENTION

The present invention relates to the field of human interaction withmobile devices, more specifically the interaction done by the automaticrecognition of barcode images captured by the camera integrated inportable devices, particularly mobile phones, smartphones, PersonalDigital Assistants PDAs, portable digital camera, among others.

The present patent application relates to a method to effect the readingof barcodes by means of portable devices with digital cameras. Thepresent invention provides a method to restore barcode images damaged bythe blurring effect due to the lack of focus of the captured images withlow cost cameras with fixed focus. The proposed method treats the imageof barcodes in order to enable correct decoding of its content.

BACKGROUND OF THE INVENTION

Currently, barcode technology is part of various sectors of thecommercial, industrial and financial. We find barcodes to identify theproducts of a supermarket, a library of books on machine of an industry,in bank bills, equipment of an organization, employee badges in acompany, etc. In general, barcodes representing unique information viathe widths of bars and spaces arranged parallel horizontal. This type ofbarcode has only one dimension and is known as one-dimensional barcodeor linear. As an example of linear barcodes has the EAN-13, Code 128,Code 39, among others. More recently emerged barcodes capable of storinginformation in vertical and horizontal and are called 2D barcode ortwo-dimensional. They can store a lot of information. As an example wecan mention the QR Code, DataMatrix, PDF417, Code 49, among others.

To perform the reading of barcode, it is used one optical readingequipment based on what is known as a scanner or barcode reader. Thepatent document US 20110068173A1 entitled SYSTEM AND METHOD FORRECOGNIZING DEFORMED LINEAR BARCODES FROM A STREAM OF VARIED-FOCUS VIDEOFRAME, published on Mar. 24, 2011, proposes a system for recognizingbarcodes. This system performs the reading of multiple frames shot bythe camera in the preview mode, wherein at least one of these tablesmust have a barcode. This system US20110068173A1 document tries toestimate the level of blurring caused by lack of focus in video frames.From this process, also trying to estimate the identity of the barcode.The method taught by document US20110068173A1 locates the regioncontaining the barcode. Another operation performed by the documentUS20110068173A1 is to create a geometrical model barcode withdeformities found barcode, which, in turn, is used to restore thebarcode. These deformities can be caused by the curvature of thepackaging, the level of tilting the camera, ambient light, among otherfactors. But the cited patent has no details on creating this geometricmodel. Document US20110068173A1 also estimates the level of blurring,where the blurring level is high, the probability of success in decodingthe barcode this frame will be low, and this, in turn, is dropped. Thisestimation is performed by calculating the differences between framesfilmed and by the same, it attempts to obtain the radius of blur presentin the barcode, where this threshold is less than the predetermineddefocus then is effected reading the barcode. The documentUS20110068173A1 does not presents a method that is able to restore theblur in the picture of the barcode, it just checks if the image has anideal focus, otherwise it discards the same. With this, only thebarcodes present in sharp images are decoded.

The international patent application WO2011011051A2, IMAGE-BASED BARCODEREADER, published on Jan. 27, 2011, discloses a method that performs thedecoding of barcodes present in images from deformable models, wherefrom the parameterization of these attempts through the maximumsimilarity obtain the digits of the barcode without turning into bitsthe image with the barcode and using all the pixel information ingrayscale. For decryption is successfully performed, it is necessarythat the location of the barcode is a reasonably precise location wheretolerance is equal to twice the width of the thinnest bar at either end.Because the sizes of labels on which barcodes are printed, do not have apattern, this method may limit the decoding of barcodes in situationswhere the widths of the bars do not match the submitted locationtolerance. Therefore, the solution proposed in International PatentApplication WO2011011051A2 is not adaptive with respect to decodingbarcode varied geometries. To perform the decoding of the barcode, themethod proposed by said patent application WO2011011051A2 tries toobtain digits comparing pixels of the image of the barcode withpredetermined templates, where they are scaled and translated toresemble those digits into analysis. The decoded digit is the one withthe highest likelihood. This process may be slower than the use offiltering, since the process of obtaining the digit corresponds to themodel is computationally expensive.

The patent document US20110007967A1 entitled: METHOD AND ARRANGEMENT FORRETRIEVING INFORMATION COMPRISED IN A BARCODE, published on Jan. 13,2011 presents a method that returns information comprised in a barcode.This method analyzes the first image to see if it has a barcode, andchecking the quality of this region where the barcode is located. If thefirst picture has a barcode, then a second image is acquired, where thequality of this image is greater than the first image and the secondregion with the barcode overlaps at least a portion of the first region.Then, the decoding is effected barcode from the second image. Thedocument US20110007967A1 tries to find the barcode on a first image withlow quality. To locate the barcode, makes up a search for rectangularregions in the image using a sliding window. The following extract isthen signatures associated with the barcode and use the method SVM(Support Vector Machine) to evaluate whether or not that window has abarcode. If this is found, then it is obtained an image with highquality. This is achieved by using quality improvement techniques suchas auto-focus, flash, lighting techniques among others. Said documentUS20110007967A1 presents no method for restoring damaged image norproposes an adaptive method for obtaining the information represented bythe barcode.

The Brazilian patent document PI0603273A, entitled System and Methodthat uses a Software to Perform Decoding Barcodes Directly on MobileCommunication Devices Equipped with Digital Camera, published on Jan.15, 2008, describes a method for decoding code bar from images capturedby digital camera of mobile communication devices. This sends the resultof decoding to a server via a network connection, which, in turn, willprovide product information related to the number represented by thedecoded barcode. If there is a network connection, so one can store theresult of the decoding for future reference. The decoding is performedin real time by analyzing the intercepted frames in the preview mode, itcan also be performed by capturing a high resolution image from thebutton press. After getting the image with the barcode starts, then itbegins the process of image processing, recognition and decoding. TheBrazilian document PI0603273A proposes methods for turning into bits theimage, correct lighting and remove noise, but does not present detailson the operation of these. This document also does not present asolution for restoration of blurred images and has no adaptive methodsfor converting image information represented in the barcode.

The American document US007237721B2 entitled: IMAGE PROCESSING FORPATTERN DETECTION published on Jul. 3, 2007 describes a method thatdetects patterns such as barcode in an image. This method finds thepoints of starting and ending patterns in a region of an image, tries toestimate the PSF (Point Spread Function—Point Spread Function (PSF) thatis the degree to which an optical system blurs (spreads) a point oflight. PSF is the inverse Fourier transform of the transfer function ofoptical frequency domain) from the start and end points detected, andrestores the region of the image using the PSF obtained. The methodproposed in this document US007237721B2 assumes that the PSF isGaussian, which is not always correct and that may limit the restorationof the barcode. Furthermore, the method performs deblurring PSFs with 1Lalong scan lines, but the PSF to restore an image should be 2L, whichlimits the power in the information retrieval barcode. Anotherlimitation of the proposed method is the use of a PSF with parametersdetermined for the picture, but the parameters of PSF should vary overthe image to perform a reconstruction closer to the ideal.

The document US2006020039A1 entitled CAMERA-BASED BARCODE RECOGNITION),published Sep. 14, 2006, discloses a system for identifying barcodeusing digital cameras. The document US2006020039A1 uses methods thatreduces noise and increases the contrast between the bars, but it doesnot describe in detail how these operations are performed. It provided amethod for locating the barcode in any orientation and position, but forthis it is necessary that the barcode is framed in the center of theimage. The method of locating document presented by US2006020039A1 isinefficient due to the high computational cost and cannot identify thebarcode in degraded images. For turning into bits the image, thedocument US2006020039A1 uses adaptive methods do not, causing thecaptured image is in optimum illumination for turning into bitsefficiently. The document also does not propose a method for restoringblurred images.

The American US20050011957A1 document entitled: SYSTEM AND METHOD FORDECODING AND ANALYZING BARCODES USING A MOBILE DEVICE, published on Jan.20, 2005, proposes a system and method for decoding barcodes. Thisdocument provides a system and method that attempts to improve the imagecaptured by a camera integrated into the mobile device and then tries toget the information represented by the barcode present in this image.The information obtained is sent to a server through a wirelessconnection, which performs a query on the content corresponding to thenumber decoded. This content, in turn, is transmitted to the mobiledevice. Document US20050011957A1 shows a block diagram describing aprocess used to correct the image and decoding the barcode, but does notdetail the procedures shown in this block diagram. The proceduredescribed in that document gets the pictures filmed by the cameraintegrated into the mobile device and from these numerous attemptsperforms decoding with different parameters until the barcode issuccessfully decoded or until all parameters are used. The documentmentions US20050011957A1 methods to remove distortions which may damagethe image of the barcode and, in turn, hinder the decoding thereof.These methods are used to: perspective correction, removal of shear;correcting problems associated with lighting, adjustable focus, amongothers. However no details are given about the operation of thesemethods, making it not be possible to make an evaluation. The documentdoes not describe US20050011957A1 step for obtaining the cut-offthreshold between black and white, which is used in the conversionprocess for the pixels bars. This document does not mention any methodto find or identify in advance the barcode before decoding, it does notdetail the process of getting the bars and it does not present a methodfor restoring blurred images.

The article “Robust Recognition of 1-D Barcodes Using Camera Phones”Wachenfeld Steffen et al., Published in 2008 in IEEE, describes analgorithm for decoding barcodes using camera phones. The algorithmdescribed by the article cited in this paragraph uses the techniques ofimage processing such as Hough transforms, wavelets to localize thebarcode patterns and morphological operations. But this article does notdescribe any method for restoring images barcode damaged by the effectof blurring, only details the procedures used in localizing the barcode,the turning into bits scan line and identify digits represented thereby.In turn, such procedures have been shown to be ineffective in decodingbarcodes present in damaged images, because the proposed procedures arenot adaptive enough, making it less efficient decoding.

The article “A Robust Method for Blind Deconvolution of Barcode Signalsand non-uniform Illumination Estimation,” L. Dumas et al., 17th IEEEInternational Conference on Image Processing (ICIP), September 2010,Hong Kong, available via the linkhttp://www.math.uvsq.fr/˜dumas/Proc8.pdf, proposes a method thatrestores the image of the barcode by removing noise and blurring usingblind deconvolution. The method proposed by the article tries to removedistortions caused by non-uniform illumination. This method is based ona genetic algorithm that combines optimization applied to discrete andcontinuous data strongly affected by the presence of noise and blurringeffect. The article describes a method for blind deconvolution based ongenetic algorithm whose solution is a target binary image, i.e., theparameters of the PSF, illumination and adjustment are also estimatedautomatically so as to minimize a cost function. Note that theuncertainty in the speed of convergence and the processing time ofgenetic algorithms may limit its applications in portable devices.

The article “Deblurring and Restoration in Barcode Signal Processing”,Todd Wittman, published in 2004 in SIAM CONFERENCE ON IMAGING PROCESSING2004, available through the linkhttp://www.math.ucla.edu/Wittman˜/thesis/bars_poster.pdf, presents atechnique of removing blurring based on the minimization of the totalvariation for restoring images of barcodes. The approach can also beclassified as blind deconvolution, because estimates of the parametersof the PSF and barcode together. The minimization of a cost function isperformed using classical optimization methods, such as Newton and“steepest descent”, which though more efficient than genetic algorithmssuffer from the limitations of local methods and also imply acomputational cost that may limit its applicability in portable devices.Another limitation of this method is to assume that the PSF is aGaussian function 1D, which is not always true.

The article “Digital Image Processing Barcode using Mobile Phones” JamesTudisco Henrique dos Santos and Almir Olivette Artero, published in 2010at the Sixth Workshop on Computer Vision, July 2010, held in PresidentePrudente, São Paulo, proposes the use of mobile phones with integrateddigital cameras to decode barcode EAN-13. The article describes a methodto locate and decode barcodes. From the Sobel operator for edgedetection is to locate the barcode and the direction of the edges fromthe gradients calculated after running this. This article presents nomethod to correct problems related to lack of focus, problems associatedwith lighting, presence of noise among others. The turning into bitsdescribed in this article may be ineffective because it has fixedthreshold and thus the results of this can be harmed under differentlighting conditions.

The article “Locating and Decoding EAN-13 Barcodes from Images Capturedby Digital Cameras” by Douglas Florian and Chai Hock, 2006, proposes avision-based technique to locate and decode barcodes. The articlementioned in this paragraph presents no technique for treatingdistortions in the image, and this has emphasized the procedure forlocating the barcode. The above documents have methods that perform thedecoding barcode and restore damaged images due to the presence ofnoise, problems related to the focal length, the problems associatedwith ambient lighting, low contrast, perspective distortions caused by,among others. Due to these problems, it is complex to perform decodingof barcodes from images captured by low cost cameras because, due to theinferior quality of these cameras, images captured easily presentproblems cited. Most of the documents cited above can perform thedecoding of barcode under certain circumstances. However, for thedecoding is successful, it is necessary that the image with the barcodeis above an acceptable threshold. Some of those documents mention theneed to carry out a pretreatment on the captured image for decryption issuccessful. But these papers do not detail the procedures necessary torestore the image, just adding a sort of black box indicating the needto improve the quality of the captured image. One of the solutions ofthe prior art is to estimate the level of blurring, where decoding isonly performed if this is smaller than a predetermined threshold.

In contrast, the present invention provides a solution to technicaldetails can restore images damaged by the effect of blurring andprovides a robust method for obtaining the number represented by thebarcode present on the captured image.

The solutions embodied by the present invention make it possible todecode the barcode present on images taken by cameras which have lowcost, and fixed focus. The present invention has a camera calibrationprocedure that creates a base of PSF's. These reflect precisely theincidence of the effect of blurring at a certain point, from a certaindistance. The PSF's this database will be used in the operation ofdeconvolution that removes the blurring effect and restore the imagewith the barcode. The present invention has adaptive methods forobtaining the bars so as to provide greater robustness under varyingconditions. The decoding of the barcode carries out a validationprocedure and blending between different results, which, in turn,enhances the probability of success in getting the numbers representedby the barcode.

SUMMARY OF THE INVENTION

The present invention utilizes techniques of image processing andcomputer vision to extract the sequence number that is contained in abarcode image. These techniques make it possible to segment the image ofthe barcode, to remove the blurring effect (blurring), to make adaptiveturning into bits image, converting the barcode digits and effecting thevalidation of the same.

The decoding method of the present invention was developed to run on aportable device with which makes capturing an image of a barcode. Fordevices that have no auto focus camera, the image acquired needs to gothrough a pre-processing to remove the blurring due to defocusing,compensating for wear caused by external interference and makingcorrections to improve the quality and making the correct decoding ofthe barcode and validation.

The present invention provides the decoding of barcodes, restoringblurred images that have barcode and decoding them in real time viaportable devices that have low cost digital camera. The system tries toachieve the ideal focused image and perform the decoding of the barcodepattern ERN-13 standard which is commonly used for identification ofproducts. This invention aims to provide a low-cost, efficient andeffective, allowing consumers to check prices anywhere they are.

The present invention also allows the user to use cameras with fixedfocus to make reading the barcode. One should frame the barcode withinthe limits of a previously positioned sight.

BRIEF DESCRIPTION OF THE FIGURES

The objectives and advantages of the invention will become more apparentfrom the following detailed description of an exemplary embodiment ofthe invention and the accompanying drawings by way of non-limitativeexample, wherein:

FIG. 1A shows an example of a blurred picture.

FIG. 1B shows an example of photo focused.

FIG. 1C presents a generic model of the portable device in which themethod works.

FIG. 2 illustrates the interface to be presented to the user.

FIG. 3 shows the barcode and the result of blurred restoration thereof.

FIG. 4 shows a focused pixel and the result of this after convolutionwith the PSF.

FIG. 5 illustrates the points to be used in extracting the PSF's.

FIG. 6 shows the result of the capture of points by the target camera.

FIG. 7 illustrates the framework of the barcode in sight of the camera.

FIG. 8 is a block diagram of the extraction process of PSF's.

FIG. 9 shows how the process of capturing the image points of the targetcamera.

FIG. 10 is a block diagram of the segmentation process of the PSF'safter capture by the target camera.

FIG. 11 illustrates the process for segmentation of PSF's.

FIG. 12 is a block diagram of the restoration process and decoding ofthe barcode.

FIG. 13 illustrates the process of convolution in the frequency domain.

FIG. 14 shows the image of a barcode after mirroring.

FIG. 15 illustrates a PSF for converting the OTF.

FIG. 16 illustrates the conversion of the image that is in the spacedomain to the frequency domain.

FIG. 17 shows the process the result of the restoration of the blurredimage.

FIG. 18 shows the process of decoding the barcode after the imagerestoration.

FIG. 19 illustrates the processing performed in the horizontal slice.

FIG. 20 shows the process of converting the pixels into bars.

FIG. 21 shows the structure of the barcode pattern EAN-13.

FIG. 22 describes the process of translation from bars to number.

FIG. 23 illustrates the process of obtaining the final result.

DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

The system of the present invention is implemented using the methodsdeveloped for restoring images of barcodes damaged by the effect ofblurring and then tries to decode them from a portable digital cameraequipped with low cost. With this, it is possible to develop a method tomake the reading of barcodes that uses images captured by a camera withfixed focus, whose quality is impaired due to a possible incorrect focallength.

Problems Solved By the Invention

Despite technological advances, mobile devices still have hardware withrelatively limited resources to implement algorithms that performadvanced calculations. Therefore, the present invention appliedprocessing techniques in order to reduce the computational cost,enabling a reduction in decoding time and minimizing the use of handheldprocessor, plus save memory, thus optimizing the use of resources byincreasing process efficiency described herein. These techniques alsoprovide an increase in strength of the barcode reader, it was possibleto increase the number of frames analyzed for the same period of time.

In portable devices with cameras without auto-focus, some images takenwith the camera cannot scan small objects clearly, as is the case withsome sources letter, where, as the camera approaches, they remain blurryand unreadable. Some packaging products have barcode so small that whenthe camera is above a certain distance, it is not possible todistinguish their bars due to insufficient amount of pixels, and whenthe camera is too close to the object can be blurred. This problem issolved by using the deblurring process described in this patent, thusobtaining an image with the focus corrected by image processing.

For cases in which the captured image of the barcode is not underappropriate conditions, could result in loss of some of its information.This type of problem can be caused by the lighting conditions of theenvironment, which may cause smudges of light, shadows excessive, lowsharpening, among others Also very common are situations where thebarcode is already damaged due to several factors, among which are: poorprint quality, surface aging, damaged packaging or container, amongothers. Aiming to improve the effectiveness of the present invention,were created methods that perform the correction of possible damageabove. These methods are based on techniques of stacking, ie calculatethe average of several adjacent lines in the image of the barcode,thereby improving the signal/noise ratio of the image.

Aiming to improve the effectiveness of the present method for readingbarcodes even under unfavorable conditions, a set of methods are appliedto the image. The barcode is divided into vertical slices, and each ofthese is processed separately. For the values of the bars, sub-regionsare defined vertical and, in turn, calculate the threshold of black andwhite for each sub-region. A set of weights which adjust the thresholdlevel of illumination present in the image are applied at threshold.With this, the effects caused by poor image quality and/or externalfactors are eliminated allowing accomplish reading the barcode.

The packaging of the products has different shapes. For example, imagesof packaging with curvilinear surface distort the size of the bars,hampering thereby decoding the barcode. To solve this problem, thepresent invention utilizes effective and efficient algorithms toestimate the size of the bars, considering a threshold based on thespecific characteristics of the dimensions of the bars used in EAN-13.

Technical/Functional Advantages of the Invention

Decoding of blurred barcodes: The use of methods to perform thedeconvolution of blurred image using the PSF or OTF allow us to estimatean image closer to the ideal for barcode decoding. Thus, removes theeffect of blurring caused by the scattering of the pixels of the imageof the barcode, increases the success rate of this method.

Versatility to adapt in different cameras: A bank's PSF can be estimatedfor each different type of camera, thus enabling the same deblurringprocess can be used with different cameras. This confers versatility tothe method, since it can be used without major changes in variousportable devices.

Possibility of using devices with low cost cameras: image correctionenables the use of devices with no auto-focus cameras or other opticallimitations because deconvolution using the PSF or OTF aims to minimizethe blurring effect caused by optical arrangement/camera. The processdescribed in this patent acts as an algorithm of digital signalprocessing that simulates the effects of corrective lenses applied tothe captured image. Thus, we obtain an image more suitable for thedecoding of barcodes, even with some limitations in image capture, thusalso enabling the use of low cost cameras.

Low computational cost: Seeking to improve the process of reading thebarcode, were used programming practices that optimize the performanceof computational instructions, such as using fixed point instead offloating point, bit manipulation used to perform some operations use theminimum possible instructions to scan or copy the pixels of an image,among others.

Reading barcode efficient and effective: Reading is efficient because itminimizes the use of computing resources, and is effective because itsucceeds in most attempts at reading, it being possible to performdecoding of barcodes with different sizes, any positioning any angle andperspective high in environments with different lighting.

Real-time processing: No need to take a photo to be taken decoding. Thesystem performs the entire procedure in real time by analyzing theframes obtained in preview mode. With this was possible to increase therobustness of the algorithm, since every time a new frame is processeduntil the read is successful. Furthermore, depending on the resolutionsupported in the preview mode by the handheld camera, it is possible tofurther enhance the effectiveness of the barcode reader.

Elimination of useless regions: regions outside the bounds found areexcluded from reading, preventing the processing of regions that do notbelong to the barcode. With this improved the efficiency of the readersince least parts of the image are analyzed, thereby diminishing theprocessing time. Efficiency is increased because elements thatnegatively influence the cutting definition of the limits of the regionof the barcode are eliminated, so that only those bars which arerelevant information, remain in processing, and thus increasing theprobability of success in decoding.

Identification Barcode: Aiming to make the decoder is not running inimages that do not have barcodes, makes up a pre-analysis within thelimits of the barcode. There currently is checked the contrast level,where, for cases in which this contrast is very low, discard immediatelythe image, making the reader does not attempt to perform any decodingthat could be unsuccessful, and with it, avoiding wasting time computingsomething that would hardly be decoded. This increases the efficiency inthe sense that, after dropping the image, the process will restart andtherefore a new framework which can have a better quality and greaterchances of decoding is provided.

Horizontal slicing Barcode: The barcode is sliced horizontally andreading is performed on each of these slices individually. With this,there was obtained a better efficacy, since the number of samples wasincreased, which led to an increase in the percentage of correctanswers. Where no success in decoding a slice, another is selected untilthe code is decoded or until all slices have been selected. The numberof slices is adjusted according to performance of reading, so thatduring the process to obtain slices of different positions and sizes,thus increasing the likelihood of successful decoding.

Adaptive method for turning into bits: phase occurs where the definitionof barcode. This step is necessary to determine a cut-off thresholdlevel to separate the white from the black level in the image of thebarcode grayscale. To this were created methods that calculate thethreshold of a flexible and adjustable. This threshold adjusts toambient lighting. This is because the slice to be decoded is dividedinto regions vertical and for each of the subregions obtained thresholdas the intensity of its pixels. Moreover, this threshold is assignedweights that adapt to the ambient light threshold. These weights aredefined through extensive testing in which it was possible to extractthe appropriate weight to each type of illumination. In the process ofreading the number of vertical slices is adjusted, computing newthresholds for black and white.

Reading barcode on non-planar surfaces: The packaging of the productshas different shapes. It can be found with packaging products withrounded surfaces, oval, or even packages which are flexible plastic bagswhich surface has a specific shape. To increase the robustness of theprocess, it created an adaptive method which enables decoding ofbarcodes on surfaces with different formats. Knowing that depend on thebar surface which is the dimensions of the bars may increase or decreasedepending on the perspective caused by the shape of the package. Becauseof this estimate a size reference bus to perform decoding in each imageregion.

Reading the barcode with perspective: to increase the effectiveness ofthe reader, techniques were developed that enable the decoding ofbarcodes in perspective. This was possible because the horizontalslicing selects slices of code that are in the same perspective favoringdecoding barcodes with high perspective.

Adaptive method for obtaining the bars: in order to increase the levelof accuracy of decoding, it created an adaptive method to convert pixelsin bars. This method attempts to compensate for problems related toleakage of pixels in its neighborhood checking for bars that were toothin or wide.

Validation result: in order to show only the correct values, the presentinvention has validation methods that verify the guard bars of the codeand check digit. Soon after the establishment of the bars is checked, ifthe guard bars are correct. After all digits have been decoded, makes upchecking the check digit. If polling is correct, the value returned isdecoded and the result is displayed.

Using hardware device with high mobility: the invention can be performedin a small and lightweight portable device that can be uploaded todifferent places without taking up much space. With this there wasobtained a portable and lightweight that performs the barcode reading inreal time, providing convenience to the user, since this will not needto move to, for example, consult product prices.

Thus, a preferred embodiment of the present invention is obtained bycounting methods to restore images blurred by the effect of blurring andperform the decoding of barcodes present on degraded images withexecution in real time from an integrated system of hardware andsoftware.

FIGS. 1A and 1B illustrate images of blurry pictures and focused,respectively.

The hardware required to realize the system of the present invention isa portable device, where its main characteristics are illustrated inFIG. 1C. The CPU (110) (Central Processing Unit) is responsible forperforming the necessary instructions for managing the camera, and makethe computations for image processing with the barcode. The camera (111)is responsible for capturing the images. The storage medium (112) servesto store information and data including the code decoded, item, amongothers. The hardware also comprises information input devices (113) andaudio components (114). Through some connectivity component (115) ispossible to make the communication between the application and a remotedatabase that has information about the products. The display means(116) is used to view the pictures captured by the camera device.

The system of the present invention begins at the moment when the useractivates it to a portable device in which it was previously installed.FIG. 2 shows a running system. After this initialization, the digitalcamera will be performed and how the preview will start displaying thepictures filmed in display (210) of the device to display a certain rate(eg 30 FPS). The embodiment of the described system has two status bars,where the upper (211) indicates the status of the barcode reader, andlower (212) presents the results of partial decoding.

The present invention has as its main goal to enter the functionality ofa reader-dimensional barcode on a portable device equipped with adigital camera with fixed focus. In turn, making it possible to create alow cost solution that is able to perform the functions of equipmentbarcode readers, commonly found in commercial establishments and areused by customers to perform the query on the value of the product.

The solution presented by the present invention has the principle effectthe deconvolution process to remove the effect caused by blurring,warping the image so that there is a degradation in the original image.

FIG. 3 shows the result of restoration image of a barcode. The barcodeaffected by the blurring effect (310) presents unfocused, and its barsno longer represent the original information. After deconvolution, thebarcode has become more clear (311) and much of the original informationis recovered.

The image restoration is closely linked to the PSF. The PSF makes thepixels of an image are focused scattered in the neighboring region,creating the effect of blur. This effect is illustrated in FIG. 4 whichshows the representation of pixel (410) and its scattering afterconvolution (411) with the PSF.

To make possible the restoration of the image, it is first necessary toestimate the PSF that caused the scattering of pixels and then do thereverse process. Therefore, we developed a method capable of extractingstatic PSF's for use in deconvolution. Said method is based on thecapture of images with black and white dots represent pixels (510). Soif this makes image capture with the camera target. The result is shownin FIG. 6, whose pixels have been scattered (610) due to incorrect focallength. Moreover, it also added noise (611) to the captured image. Thisspreading (610) constitutes PSF, which should be targeted for use in themethod of the present invention. The PSF that caused the blurring can bechanged according to the distance from the camera to the target object.Seeking to reduce the number of PSF's to be used, it created a sight(710) to frame the barcode, making the camera runs about the samedistance from the source to the barcode filmed or photographed.

FIG. 7 illustrates how the code should be framed in the sight.

To restore the image and make the decoding, first it is necessary toperform the camera calibration to obtain the PFS's. The steps for thecalibration are shown in FIG. 8. The calibration consists in a processof capturing images of points at distances previously established, and,in turn, target points of the captured images.

As mentioned previously, to extract the PSF's, it is necessary to firstcreate an image (810) with black and white dots representing the pixels.Before making the capture of images, you need to first stipulatedistances (811) wherein the portable device will be the same. Thisdistance should reflect the position where the user fits the barcode insight, because it is measured as the width of the barcode. As thebarcodes on standard EAN-13 does not have a standard size, then you needto take pictures at different distances.

FIG. 9 illustrates the capture of the sizes of dots to two differentbarcodes, where is positioned the camera (910) framing an image whitepoint (911) that has been resited depending on the size of a particularbarcode. To capture this image is then necessary to fit it in the sight(912), simulating the user to frame the barcode. It appears, then,another image of white dots (914) which resines the width of anotherbarcode. In turn the camera should be repositioned (913) to frame theimage. With this, it creates a base PSF's for different sized barcode.

Capturing these images should be made from the camera target (812).Models of different cameras can have different focal distances, makingit necessary to get the base PSF's data concerning particular camera.

After capturing the images, it's necessary to segment the PFS. For this,first performs turning into bits to separate the image background imageof white pixels (1010). So, it makes up a sweep on the digitalizedimage, searching for areas with high levels of intensity (1011). Thenthe white spots, which are now scattered in their neighborhoods, are cutand separated into individual images (1012). This image with the PSF isadded to an initial database (1013) for further analysis.

FIG. 11 shows the process of extracting the PSF's, where the capturedimage (1110) is digitalized (1111) by creating a mask of the regionswith the probable positions of the PSF's. Then, a copy only the valuesof PFS's (1112), excluding the background image. Then cut up each one ofPSF's (1113) present the images to store them separately on an initialbasis. This initial basis is analyzed to select the best PSF's thatrestores the images (814). This check is made by a method that attemptsto decode multiple barcodes applying blur PSF's base to restore theoriginal image. The PSF's in which decoding is performed successfullydecoded and whose numbers are exactly equal to the numbers encoded bybarcode will be selected.

According to the present invention, the deconvolution performed toremove the blurring is implemented in the frequency domain. Therefore,to avoid unnecessary processing, becomes the PSF that is in the field ofspace OTF, used to do convolution and deconvolution in the frequencydomain. Then, first normalize the PSF's from a previously establishedrange and performs the conversion to OTF. This process will be explainedin detail below. After calibration of the camera and obtaining therespective OTF's for restoring the image, performs the process forreading the barcode itself.

FIG. 12 illustrates the steps performed in the portable device when itstarts reading the barcode.

The step “capturing the image with the barcode” (1210) occurs in realtime is the time in which a filmed frame by the camera is selected andsent for decoding. In the system which was developed as one of thepossible embodiments to the present invention, we created a button thatis used to initialize capturing frames. At this time, the user must fitthe barcode on sight as shown in FIG. 2. The capture of the images willbe executed until the barcode is successfully decoded, or until thetimeout is completed.

After capturing of the frame, begins at step “restores image” (1211). Inthis step, it selects one OTF in the database and form deconvolution. Todescribe the process of image restoration, it is necessary to describemathematically how does the scattering of pixels and deduct it inverselyuntil the results of the present invention.

The effect of blurring can be modeled mathematically as a convolutionbetween the pixels and the PSF. This operation is shown in detail below:

b(x, y)=psf(x, y)*i(x, y)   (1)

Equation (1) shows the mathematical model that describes how thescattering of the pixels occurs, and the symbol “*” is used to definethe convolution operation of the psf (x, y) and i (x, y) at the point(x, y), where i (x, y) is the ideal image.

The result of this convolution is the defocused image, represented byb(x, y). So, to remove the blurring effect, we must make the reverseprocess, where, from the blurred image, we obtain the original image.

i(x, y)=psf⁻¹(x, y)*b(x, y)   (2)

Equation (2) shows how it would restoration image using convolution,where i (x, y) is the ideal focused image, psf−1 (x, y) is the inversePSF b (x, y) represents the image blurring. Then, to remove the effectof blurring from the convolution operation, it is necessary to estimatethe inverse PSF, and through the application of the convolution with theblurred image, it is possible to remove the scattering displayed inpixels.

It is known that the convolution of two functions in the space domainbecomes transformed the product between these two functions in thefrequency domain. Therefore, based on this property, the deconvolutionin the frequency domain is given by dividing the transformed twofunctions. That is, the division between the transformed image and theblurred PSF will result in a transformed image ideal focused.

The transformed image in the frequency domain is obtained from adiscrete Fourier transform. In turn, it applies the inverse Fouriertransform to return to the spatial domain.

$\begin{matrix}{{F( {u,v} )} = {\frac{1}{MN}{\sum\limits_{x = 0}^{M - 1}{\sum\limits_{y = 0}^{N - 1}{{f( {x,y} )}{\exp \lbrack {- {{j2\pi}( {\frac{ux}{M} + \frac{vy}{N}} )}} \rbrack}}}}}} & (3)\end{matrix}$

The discrete Fourier transform is given by equation (3), where u=(0, 1,2, . . . , M−1) v=(0, 1, 2, . . . , N−1).

$\begin{matrix}{{f( {x,y} )} = {\sum\limits_{u = 0}^{M - 1}{\sum\limits_{v = 0}^{N - 1}{{F( {u,v} )}{\exp \lbrack {{j2\pi}( {\frac{ux}{M} + \frac{vy}{N}} )} \rbrack}}}}} & (4)\end{matrix}$

Equation (4) corresponds to the inverse Fourier transform, for x=(0, 1,2, . . . , M−1) y=(0, 1, 2, . . . N−1), where Δu 1/MΔ×e=Δv=1/NΔy.

The method of the present invention uses the algorithm of fast Fouriertransform (FFT—Fast Fourier Transform) to calculate the discrete Fouriertransform, since through this it is possible to reduce the complexity ofN² to N log₂ N operations and in turn reduce the cost computationalincurred in calculating transformed.

Equations (5) to (8) present a mathematical model used to cause blurringeffect from frequency domain.

I(u, v)=FFT(i(x, y))   (1)

PSF(u, v)=FFT(psf(x, y))   (2)

B(u, v)=I(u, v)PSF(u, v)   (3)

b(x, y)=FFT⁻¹(B(u,v))   (4)

Equation (5) calculates image transformation from the space domain (orspatial domain) to the frequency domain using the fast Fouriertransform, represented by the FFT, where i (x, y) represents the idealimage focused in the area of space and I (u, v) is its image in thefrequency domain. Equation (6) performs the same calculations of theequation (5) where psf (x, y) represents the point spread function inthe space domain and PSF (u, v) is the same in the frequency domain.

Equation (7) adds the blurring effect of multiplying the PSF (u, v) I(u, v). Finally, the image is transformed into the domain of space againfrom the algorithm inverse fast Fourier transform, as shown in equation(8) by FFT−1, returning, in turn, the blurred image to the field Space b(x, y).

The process described in the previous paragraph is illustrated in FIG.13. The barcode (1310) and PSF (1311) are in the space domain. Calculatethen their Fourier transforms (1312), obtaining the images in thefrequency domain. The magnitude of the spectra of the transformedBarcode (1313) and PSF (1314) with origin shifted to the center arepresented and represent the images in the frequency domain. Transformsare multiplied (convolution operation in the frequency domain) (1315)and inverse Fourier transform (1317) is applied on the result, obtainingturn the blurred image in the space domain (1318).

From the understanding of this information, then developed a processthat performs a set of operations, where from the blurred image we tryto achieve an optimal image focused. These operations are based on imagedeconvolution in the frequency domain, dividing the transformed imageblurred by its scattering transform function.

Equation (9) mathematically describes the deconvolution operationdescribed in the preceding paragraph.

$\begin{matrix}{{I( {u,v} )} = \frac{B( {u,v} )}{{PSF}( {u,v} )}} & (9)\end{matrix}$

The methods for processing images in order to obtain the ideal focusedimage:

N=2^(max(ceil(log) ² ^((col)),ceil(log) ² ^((lin))))   (10)

To reduce the complexity and increase the speed of calculation ofprocessed, the algorithm of fast Fourier transform used working withvectors or matrices whose dimensions are powers of 2. Equation (10) isused to obtain the proper dimensions for calculating the fast Fouriertransform, where col is the number of columns, row the number of linesof the image and a function ceil is a value used for rounding up. Thus,the matrix used in calculating transformed to have equal dimensions N×Nwhere N equals the highest power of base two rounded up between thepowers of the number of columns and number of rows. After obtaining theappropriate dimensions for calculating the transformed creates an arraywith these dimensions and copy to the image pixels for this matrix,which may result in a surplus space due to the difference between thedimensions. In general, if the space fills with excess zeros, but tominimize edge effect often caused by the Fourier transform image, copiesthe image to the center of the new matrix (1410) and makes up mirroringthis Up (1411) and down (1412).

FIG. 14 illustrates an input image for the fast Fourier transform. Thepresent invention has a base which is extracted PSF's for the cameratarget. To decrease the processing, the PSF's are stored in thefrequency domain in the form of an OTF. Thus, it is possible toeliminate the step of converting the OTF PSF for the portable device,thus reducing the computational cost while restoring the image.

FIG. 15 illustrates the process of converting PSF for OTF. The PSF(1510) is copied to a new image (1511) with appropriate dimensions(1512) for calculating transformed. Then, the displacement is effectedcircular (1513) with the offset value equal to half the dimension of thePSF multiplied by −1, repositioning the pixels in the new image aspresented in 1514. Following the algorithm of fast Fourier transform(1515) runs, transforming the matrix of the space domain to thefrequency domain, where the result will be the OTF, whose spectrum isshown for 1516. This OTF is stored and loaded onto the portable deviceto be used in image restoration while reading the barcode.

The process performed to calculate the transformed image in thefrequency domain is presented in FIG. 16. The image of the barcode(1610) is copied into the center of an image with the appropriatedimensions to calculate the transform (1611), then it performs mirroring(1612) this. Then calculate the transform (1613) of new image, theresult of which spectrum is shown by 1614.

$\begin{matrix}{{I( {u,v} )} = \frac{B( {u,v} )}{{OTF}( {u,v} )}} & (5) \\{{I( {u,v} )} = \frac{\overset{\_}{{OTF}( {u,v} )} \times {B( {u,v} )}}{\overset{\_}{{OTF}( {u,v} )} \times {{OTF}( {u,v} )}}} & (6) \\{{I( {u,v} )} = \frac{\overset{\_}{{OTF}( {u,v} )} \times {B( {u,v} )}}{{{{OTF}( {u,v} )}}^{2}}} & (7)\end{matrix}$

In equations 11 to 13, the symbol×represents the conventional arithmeticmultiplication. After obtaining the transform of the PSF and thetransformed image blurred, is then performs the operation ofdeconvolution that may be modeled mathematically as equation (11), whereit is given as a division between OTF (u, v) and B (u, v). Based on theproperties of complex numbers, multiplies the numerator and denominatorby the conjugate of the OTF (u, v) as shown in equation (12). Equation(13) changes the denominator for |OTF (u, v) |², for a complex numbermultiplied by its conjugate equals its squared norm. In equation (11)may occur the problem of dividing by zero or by many small numbers. Toeliminate this problem, one needs to determine a number that afterdividing the processed does not result in inconsistent values. Thedenominator of the equation (13) is a real number, which facilitatescomparisons made to obtain this number.

λ=max( |OTF(u, v)×B(u, v)|)×√{square root over (ε)}  (14)

The equation (14) calculates the limit number for the denominator, whichis represented by λ. This value is the highest value of the numeratormultiplied by the root of ε, where the value of ε is a small number thattends to zero.

$\begin{matrix}{{D( {u,v} )} = \{ \begin{matrix}{{{{OTF}( {u,v} )}}^{2},} & {{{para}\mspace{14mu} {{{OTF}( {u,v} )}}^{2}} \geq \lambda} \\{\lambda,} & {{{para}\mspace{14mu} {{{OTF}( {u,v} )}}^{2}} < \lambda}\end{matrix} } & (15) \\{{I( {u,v} )} = \frac{\overset{\_}{{OTF}( {u,v} )} \times {B( {u,v} )}}{D( {u,v} )}} & (16)\end{matrix}$

The final mathematical representation for the operation of deconvolutionused by the present invention for removing the blurring effect is givenby equation (16), where the denominator D (u, v) is obtained fromequation (15). This equation returns λ, where |OTF (u, v)|2 is smallerthan λ, else returns |OTF (u, v)|2. Through this process, it avoidsdivision by zero or very small numbers. The result of this equation,represented by I (u, v) is the converted image restored.

After deconvolution in the frequency domain, becomes the ideal image ofthe targeted frequency domain back to the spatial domain, through thealgorithm of fast inverse Fourier transform, thus getting the idealimage focused in the space domain.

FIG. 17 shows the result of image restoration. The image of the barcode(1710) is prepared for the operation of deconvolution (1710) in thefrequency domain. This operation results in a transformed image withoutblurring effect. In 1712, the spectrum is displayed with origin at thecenter of the resulting transformed the operation of deconvolution.Through the algorithm fast inverse Fourier transform (1713), becomes theresulting transformed to the domain of space, transforming it into thefocused image (1714). Then just cut up the downtown area (1715) with thebarcode restored (1716). This in turn is used as an input parameter tothe algorithm for decoding barcode.

After the image restoration, step “decoding barcode” (1212) starts,which receives the image of the barcode restored and tries to decodethis. This process is divided into sub-steps that perform imageprocessing to return the numbers represented by the barcode in theimage. This process is detailed by the flowchart in FIG. 18 and isdescribed below.

The barcode (1810) present in the restored image is sliced horizontally(1811) and each of these slices is processed individually. The amount ofhorizontal slices is one of the inputs of barcode reader, and this isadaptive, ie depending on the implementation of barcode reading thenumber of horizontal slices may be modified as to increase the amount ofdecoded digits. Each slice will return an array with the digits decoded,and the size of this vector is equal to the number of digits of thebarcode in EAN-13 standard. The horizontal slicing is shown in 1812.

$\begin{matrix}{H_{s} = {{floor}( {H \times \beta} )}} & (8) \\{N_{s} = {{floor}( \frac{H - H_{s}}{H_{s}} )}} & (9) \\{{\Delta_{s} = {k \times H_{s}}},{{{com}\mspace{14mu} k} = 0},1,2,3,\ldots \mspace{14mu},N_{s}} & (10)\end{matrix}$

Equation (17) calculates the height of the horizontal slice where bloomis a function that rounds down. The size of the slice is represented byHs, which is a percentage defined by β, H, which represents the imageheight of the barcode. The number of horizontal slices is calculatedusing equation (18) represented by Ns. The offset between lines of theimage of the barcode, represented by Ds is calculated from equation(19).

The barcode pattern in EAN-13 has only one dimension, ie the informationis arranged in a line therefore needs to convert the horizontal slice,which is a two-dimensional array in a vector. This process is donethrough vertical stacking (1813) that transforms a two dimensionalmatrix in a vector (1814), and the values of the vector corresponds tothe weighted average of the columns of the first matrix.

$\begin{matrix}{{\mu_{s}(x)} = {\frac{1}{H_{s}} \times {\sum\limits_{y = \Delta_{s}}^{\Delta_{s} + H_{s}}{i( {x,y} )}}}} & (20)\end{matrix}$

The vertical stacking is calculated as is shown by equation (20),converting the two-dimensional array represented by i (x, y) in vectorμS (x). This vector stores the weighted averages of the columns betweenthe Δs and Δs+Hs, where Δs is the vertical displacement and Hs is thenumber of rows of horizontal slices.

The next process is digitalizing (1815) the vector resulting fromvertical stacking. This process will convert the values of this vector,which are among the range of 0 to 255, in just two values, black “0” andwhite “1” (1816). For this, one needs to define a threshold thatseparates the range of values that match the values that correspondblack white. Due to adverse conditions such as the presence of imagenoise and lighting levels from the external environment, the range ofvalues of the vector may vary affecting the result of turning into bits.Because of this, the vector is divided into regions and computes athreshold for each of these regions. The threshold is a weighted averageof the region multiplied by a weight belonging to a set of weightspreviously defined. These weights are used to adjust the threshold fordifferent levels of illumination. If the value of the vector is lessthan the threshold, then white is assigned “0” otherwise, white isassigned “1”, resulting in another vector whose values are between 0 and1.

$\begin{matrix}{w = \{ {w_{1},w_{2},\ldots \mspace{14mu},w_{n}} \}} & (11) \\{W_{r} = \frac{W}{R_{n}}} & (12) \\{l_{r} = {( {\frac{1}{W_{r}} \times {\sum\limits_{x = x_{r}}^{W_{r} + x_{r}}{\mu_{s}(x)}}} ) \times w_{i}}} & (13) \\{{b(x)} = \{ \begin{matrix}{1,} & {{{para}\mspace{20mu} {\mu_{s}(x)}}>=l_{r}} \\{0,} & {{{para}\mspace{20mu} {\mu_{s}(x)}} < l_{r}}\end{matrix} } & (14)\end{matrix}$

In equations 21 to 24 the method is shown turning into bits vector.Equation (21) represents the set of weights used in calculating thethresholds. In equation (22) calculates the size of the regions in whichthe vector will be divided, where N represents the size of the vector,the amount Rn regions to be used and Wr is the size of the regions. Thevalue of Rn is adaptive and can be changed to improve the result ofdecoding. The threshold is computed by equation (23), where xr is thebeginning of region r represents a weight Wi in the set of weights andIr represents the threshold calculated for the region r. Finally, theequation (24) calculates the digitalized vector, where b (x) is white,“1” for μS (x) greater than or equal to LR or black “0” for μS (x) lessthan Ir.

The procedure described in the previous paragraph is detailed in FIG.19. A horizontal slice (1910) is stacked vertically (1911) converting itinto a vector. Then, one obtains the weight (1912) to be applied incalculating the threshold by dividing the array into regions (1913) andcalculates the threshold for turning into bits (1914). After this, it ispossible to identify the barcode and compute these digits (1915). Thedigits obtained are stored (1916) for later use. This process isrepeated until all weights set of weights are used (1917). The step forconverting pixels bars (1817) receives the digitalized vector with pixelinformation and converts it to a vector representation of the barcodepattern on EAN-13, which is formed by four bars of different levels:“thin”, “medium: 2 thin bars”; “wide: 3 thin bars”, “very large: 4 thinbars.” With this, we created a method that tries to estimate how manypixels each bar has. This method tries to find the amount of pixels of athin bar and from this set the size of the other bars.

The digitalized vector (2010) is converted into two vectors, one of thebars with sizes in pixels and a second vector containing the color bars(black, “0” or blank “1”) (2011). These vectors are utilized in thecalculations performed during the conversion of pixels to bars. Toobtain the correct sizes of each bar needs to first find the start ofbarcode. It is known that the barcode in the standard EAN-13 starts withthree thin bars interspersed between black and white. It is thebeginning of the barcode when it finds a black bar “0”, a white bar “1”and a black bar “0”, whose biggest difference between the sizes of thebars is less than a threshold previously stipulated.

FIG. 21 shows the structure of a barcode on EAN-13 standard. In thispattern, the first three bars (2110) are used to identify the beginningof the barcode and the last three bars (2114) identify the end of thebarcode. The five centers of the bars (2112) divide the barcode into twogroups where the first group (2111) seven figures represents the initialbarcode, and the first digit (2115) is defined by parity between thebars the first group. The second group (2113) represents the last sixdigits of the barcode. Each digit barcode is represented by four bars.

Thus, to effect the reading of the barcode, are located the first barsrepresenting the beginning of the barcode. This procedure uses thevector with the sizes of bars in pixels and vector with the colors ofthe bars, shown in 2011. It is known that the beginning of the barcodepattern EAN-13 is represented by three thin bars with black, white andblack, respectively (2110). Therefore, they created an adaptive methodthat attempts to locate this bar pattern from the difference between thelargest sizes of three continuous bars. If this difference is less thanthe threshold used to represent the shortest distance between the sizesof the bars and the bar colors match the colors of the bars of thepattern that represents the start, then, means that the start of barcodewas located.

$\begin{matrix}{{{M( {v,x} )} = {\underset{i \neq j}{Max}( {{{v( {x + i} )} - {v( {x + j} )}}} )}},{\forall i},{j{i \in {\{ {0,1,2} \} e\mspace{14mu} j} \in \{ {0,1,2} \}}}} & (25)\end{matrix}$

The equation (25) calculates the difference between bars, where v is thevector with the size of the bars in pixels x is the displacement vector.If the value returned by M (v, x) is smaller than the lower thresholddifference, then the first group of barcode begins at position x×v 3vector. Therefore, begins translating the bars for the digits of thebarcode from the position vector x 3+to the size in pixels of the barsand with the vector containing the colors of the bars.

To perform the translation of the digits, the vectors are traversedevery four bars to complete the six digits of the first group (2110).Then the five bars are discarded from the center (2112), and makes upthe translation of the last six digits of the second group (2113). Inthe translation of the first digit is calculated also the parities ofthese, representing the first digit barcode (2115).

FIG. 22 shows an example of implementation of the conversion process inthe bars digits. In 2210, the bars are presented that represent thenumber four in the second group. In this process uses a vector with thesize of the bars in pixels (2211) and a second vector identifying thecolors of the bars. From these, it created an adaptive method that aimsto convert the pixels in only four of bar levels: “1” for thin bar; “2”to mean bar; “3” for wide bar, “4” for very large bar.

It is known that each digit of the barcode is represented by seven thinbars. Therefore, the threshold used for setting the size of the bars isof thin bar (2212). The threshold size or thin bar is obtained throughthe sum of the length of the bars in pixels divided by seven (2213).Even after the restoration process of the image, you can find picturenoise. These in turn affect the final result of decoding the barcode. Tocircumvent this problem, the method that converts the pixels bars to tryto adjust the threshold at the time of translation. This adjustment ismade by applying a set of weights threshold (2214), generating differentthreshold values (2215).

The conversion of pixels to bars is done by dividing the size of thebars in pixels by thresholds obtained after the application of theweights (2216). The result is a matrix whose rows represent the resultsof different thresholds and the columns represent the sizes of the bars(2217).

Due to the presence of noise, there may be cases wherein the process forobtaining the sizes of the bars is damaged, causing some bars to havemore pixels, and in turn, other pixels have the least. That is, somebars will be more thin and other extremely wide. To solve this problem,we developed a method that tries to restore the size of the bar. Thismethod tries to identify the bars that were very thin, seeking valueswhich result from the division of the sizes of bars in pixels by thethreshold was lower than 0.5. It is then added 0.5 to bar which was verythin and subtracts 0.5 from neighboring bars. This process is describedin 2218 and an example of its result is presented in 2219.

The following procedure is the conversion of the resulting values tointegers. For this round if the values (2220), leaving only the integerpart (2221). Then, converting it to the values obtained in bars, whichare represented by a combination of 0's (white bars) and 1's (blackbars). This combination has seven elements, where each of these is athin bar. We then obtain a list of these combinations (2223) and thenperform decoding up (2224) the bars in search of the number representedby the latter. If successful, obtaining a list with numbers decoded.Should one or more digits are not decoded if returns a value of −1 foreach of these respectively (2225).

The result of the process described in the preceding paragraph is a listof probable numbers (2225) which may be represented by the bars decoded.So, from this list, we try to estimate the number most likely to berepresented by the bars decoded. This is done by calculating thedominant digit (2226), which is the number other than −1 more frequentlyin the list of numbers decoded, as presented in 2227. With this, itfollows then step of translating the digits of the barcode (1819).

The process of turning into bits (1815), conversion of pixels for bars(1817), and translation of the digits of the barcode (1819) generates avector with thirteen positions that contains the possible values of thenumbers represented by the barcode. These operations are performed ineach of horizontal slices, generating in turn a vector of results foreach of the slices. These vectors are used to construct a histogram ofdigits (1820), which is used to obtain the number represented by thebarcode.

By the histogram of digits that is generated from the decoding ofhorizontal slices attempts to reach an optimal outcome. FIG. 23illustrates the process of obtaining the histogram digit and generatingthe final result.

Each horizontal slice of the code (2310) is decoded separately returninga sequence of digits decoded for each slice (2311). In case of failurein decoding single digits, is assigned the value −1 (minus one) inposition.

To obtain the final value is calculated a histogram of digits (2312).The frequency of this histogram is affected by the amount of numbersreturned successfully during decoding.

$\begin{matrix}{{s(x)} = \{ \begin{matrix}{0,} & {{{para}\mspace{14mu} {u_{i}(x)}} = {- 1}} \\{1,} & {{{para}\mspace{14mu} {u_{i}(x)}} \neq {- 1}}\end{matrix} } & (15) \\{{{inc} = {\sum\limits_{x = 0}^{n}{s(x)}}},{{{onde}\mspace{14mu} n} = 12}} & (16) \\{{{H( {y,x} )} = {{H( {{u_{i}(x)},x} )} + {inc}}},{{{se}\mspace{14mu} {u_{i}(x)}} \neq {- 1}}} & (17)\end{matrix}$

The process for obtaining the histogram of digits is presented inequations 26 to 28. Equation (26) returns 0 (zero) if the digit is equalto −1 (minus one), otherwise returns 1 (one), where u_(i) is a vectorwith the sequence of digits returned in decoding a slice whose index isrepresented for i. Equation (27) calculates the increment of thehistogram, where it is equal to the number of elements ui different from−1 (minus one). Equation (28) increments a value in the histogramrepresented by H, where y is digit to increment x is the position ofthat digit in the barcode. The histogram will have value only if theupdated value ui at position x is different from −1 (minus one).

$\begin{matrix}{{R(x)} = {\underset{y \in {\{{1,2,{3\mspace{11mu} \ldots}\mspace{14mu},9}\}}}{\arg \; \max}{H( {y,x} )}}} & (29)\end{matrix}$

After calculating the histogram of digits, then we obtain the numbersrepresented by the barcode (1821) (2313), which are those mostfrequently in the histogram of digits. This process is shown by equation(29) where y represents the numbers barcode exact position thereof. Theend result of decoding is shown in equation (29) for R, stored in avector with thirteen (13) positions.

The next process is the validation of the result (1822), wherein fromthis analyzes the sequence of numbers returned in decoding in order toverify that this is a valid string. For this, first it is verifiedwhether all the digits have been decoded correctly. If all numbers havebeen successfully decoded, then it calculates check digit. If thechecksum is correct, then the decoded string is valid.

In the event of an error, a new frame is captured and it carries out theprocess again until the barcode is successfully decoded or until endingreading timeout.

When the decoding a barcode is successfully performed, then the step“displays the result” (1213) begins. At this step, the decoded code isdisplayed on the portable display. From this result, it is possible tocarry out research on the selling points that provide the productidentified by the barcode decoded but also see the price of this.

Although a preferred embodiment of the present invention is shown anddescribed, those skilled in the art will understand that variousmodifications may be made without departing from the scope and spirit ofthe invention as defined in the appended claims.

It is also expressly stated that all combinations of elements whichperform the same function in substantially the same way to achieve thesame results are within the scope of the invention.

1. Method for restoring images of blurred barcodes comprising the stepsof: Capturing the image with the barcode (1210), in which a picture shotby the camera is selected and sent for decoding; After capturing theframe, starting the step restores the image (1211), in which it selectsone OTF in the database and makes deconvolution; After imagerestoration, initiating the step of decoding barcode (1212) whichreceives the image of the barcode restored and tries to perform thedecoding thereof, said process being divided into sub-steps that performimage processing to return the numbers represented by the barcodepresent in the image; When the decoding a barcode is successfullyperformed, then start step “displaying the result” (1213),
 2. Methodaccording to claim 1, wherein it is based on the capture of images withblack and white dots representing pixels (510) to thereby accomplish thecapture of said image with a target camera.
 3. Method according to claim1, wherein the pixels are spread (610) due to incorrect focal length,said spreading (610) consisting of the PSF, which is to be segmented foruse.
 4. Method according to claim 1, wherein it further comprises thefollowing steps: The captured image (1110) is digitalized (1111) bycreating a mask of the regions with the probable positions of the PSF's;Then, copying only the values of PFS's (1112), excluding the backgroundimage; Then, cutting each one of PSF's (1113) present the images tostore them separately in an initial base, said base being analyzed toselect the initial PSF's that best restore the images (814); Saidverification being made by a method that attempts to decode multiplebarcodes applying blur PSF's base to restore the original image; whereinthe PSF's in which decodings are performed successfully and whosenumbers are exactly equal to the numbers encoded by the selectedbarcodes.
 5. Method according to claim 1, wherein the size of the barsis estimated considering a threshold based on the specificcharacteristics of the dimensions of the bars used in EAN-13.
 6. Methodaccording to claim 1, wherein the deconvolution of blurred image usingthe PSF or OTF allows to estimate an image closest to the ideal fordecoding the barcode, removing thus the blurring effect caused by thepixels spreading of barcode image, increasing the success rate of themethod.
 7. Method according to claim 1, wherein it comprises a procedurefor camera calibration which creates a base PSF's, reflecting the impactof the blurring effect at a given point from a given distance, and thePSF's of this database used in that operation deconvolution that removesthe blurring effect and restore the image with the barcode.
 8. Methodaccording to claim 1, wherein it comprises extracting static PSF's foruse in said deconvolution.
 9. Method according to claim 1, wherein abank of PSF's can be estimated for each different type of camera, thusenabling the same deblurring process can be used with different cameras.10. Method according to claim 1, wherein the image correction enablesthe use of devices with no auto-focus cameras or other opticallimitations, the deconvolution using the PSF or OTF minimizing theeffect of blurring caused by arrangement optical/camera.
 11. Methodaccording to claim 1, wherein it performs real-time processing,analyzing the frames obtained in the preview mode.
 12. Method accordingto claim 1, wherein the regions outside the limits encountered areexcluded reading, avoiding processing regions which do not belong to thebarcode.
 13. Method according to claim 1, wherein the barcode is slicedhorizontally and reading is performed on each of these individualslices.
 14. Method according to claim 1, characterized by the fact thatthe number of slices is adjusted according to performance of reading, sothat during the process to obtain slices of different positions andsizes, thus increasing the success probability in decoding.
 15. Methodaccording to claim 1, wherein, in the definition of the barcode, it isnecessary to determine a cutting threshold to separate the white levelof the black level in the barcode image in grayscale.
 16. Methodaccording to claim 1, wherein said threshold adjusts to the ambientlighting.
 17. Method according to claim 1, wherein said slice to bedecoded is divided into vertical regions and for each of the subregions,it is obtained a threshold in accordance with the intensity of itspixels.
 19. Method according to claim 1, wherein it performs thedecoding of barcodes present on images taken by cameras having low costand fixed focus.
 20. Method according to claim 1, wherein it isnecessary to obtain the FHP's data base concerning a particular cameraaccording to the different models of cameras having different focallengths.